In large-scale distributed file systems, efficient meta- data operations arecritical since most file operations have to interact with metadata serversfirst. In existing distributed hash table (DHT) based metadata managementsystems, the lookup service could be a performance bottleneck due to itssignificant CPU overhead. Our investigations showed that the lookup servicecould reduce system throughput by up to 70%, and increase system latency by afactor of up to 8 compared to ideal scenarios. In this paper, we presentMetaFlow, a scalable metadata lookup service utilizing software-definednetworking (SDN) techniques to distribute lookup workload over networkcomponents. MetaFlow tackles the lookup bottleneck problem by leveragingB-tree, which is constructed over the physical topology, to manage flow tablesfor SDN-enabled switches. Therefore, metadata requests can be forwarded toappropriate servers using only switches. Extensive performance evaluations inboth simulations and testbed showed that MetaFlow increases system throughputby a factor of up to 3.2, and reduce system latency by a factor of up to 5compared to DHT-based systems. We also deployed MetaFlow in a distributed filesystem, and demonstrated significant performance improvement.
展开▼
